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Abstract 

This paper deals with the specification and the implementation of 
multi-level agent-based models, using a formal model, IRM4MLS (an In- 
fluence Reaction Model for Multi-Level Simulation), based on the Influ- 
ence Reaction principle. Proposed examples illustrate forms of top-down 
control in (multi-level) multi-agent based-simulations. 

Keywords: multi-level simulation, influence reaction model, micro-macro 
link, cellular automata. 

1 Introduction 

Four main theoretical issues emerge in the literature on multi-leveQ agent-based 
modeling: the conception of a meta-model allowing a non ambiguous charac- 
terization of a multi-level model at the conceptual level [TSl [231 HSl 123], the 
introduction of a dynamic level of detail [25l [31] , the detection and reification of 
emergent phenomena [21 [3l|5l [21] [301 131! ^^'^ the representation of aggregated 
entities [27]. This paper focuses on the first one, with respect to the Influence 
Reaction (IR) principle, shortly, action as a two step process: (1) agents pro- 
duce "influences", i.e., individual decisions, according to their internal state and 

^The term multi-scale may also be found. Intuitively, a level and scale are similar concepts 
that both mean viewpoint. However, this notion should be clarified. In the following we 
assume that two agents are not at the same scale iff they represent processes that have 
different spatial and/or temporal extents. Two agents are not in the same level iff they cannot 
interact directly, i.e., with a single interaction function. It should follow from the previous 
definitions that it exists multi-scale models that are mono-level and conversely. 



perceptions (2) the system "reacts", i.e., computes the consequences of influ- 
ences, according to the state of the world [TUj. This model has been extended in 
several ways, and notably for multi-agent based-simulation (MABS), by adding 
an explicit representation of time [R]- An IR-based meta-model, IRM4MLS 
(an Influence Reaction Model for Multi-Level Simulation), and its Java imple- 
mentation are introduced in the section [2] Using, and then extending a simple 
example: the Conway's game of life, two toy-models of increasing complexity are 
presented in the section [sj illustrating forms of top-down control in (multi- level) 
MABS. Results are discussed in the section IH 

2 IRM4MLS: an Influence Reaction Model for 
Multi-Level Simulation 

In this section, IRM4MLS, an Influence Reaction Model for Multi-Level Sim- 
ulation, is introduced [55]. It extends IRM4S (an Influence Reaction Model for 
Simulation) in order to deal with multi-level models |19J. From a technical perspec- 
tive, levels can be viewed as interacting IRM4S simulation^ 

A multi-level model is defined by a set of levels L and a specification of the relations 
between levels. Two types of relations are specified: an infiuence relation (agents in a 
level / are able to produce infiuences in a level I' ^ I) and a perception relation (agents 
in a level I are able to perceive the dynamic state of a level I' ^ I), represented by 
directed graphs denoted respectively < L,Ei > and < L,Ep >, where Ej and Ep 
are two sets of edges, i.e., ordered pairs of elements of L. Influence and perception 
relations in a level are systematic and thus not specified in Ei and Ep (cf. eq. [T] and 
[2|. 

The in and out neighborhood in < L,Ej > (respectively < L,Ep >) are denoted 
Nf and N/ (resp. Np and Np) and are defined as follows: 

G L,NY{1) (resp. Np{l)) = {1} U {l' e L : I'l e Ei (resp. Ep)}, (1) 

V/ G L,Ntil) (resp. N+{1)) = {1} U {/' G L : //' G Ei (resp. Ep)}, (2) 

The set of agents in the system at time t is denoted A{t). VZ G L, the set of agents 
belonging to Z at t is denoted Ai{t) C A{t). An agent belongs to a level iff a subset of 
its physical state (pa belongs to the state of the level. Thus, an agent belongs to zero, 
one, or more levels. An environment models the natural dynamics of level properties 
and can be shared by different levels (flg. [l]). 

The dynamic state of a level I £ L at time t, denoted 5'{t) G A', is a tu- 
ple < a\t),'y'{t) >, where a'{t) G and ^{1) G F' are the sets of environmen- 
tal properties and influences of I. The behavior of an agent a £ Ai is deflned as 
Decision^ o MemorizatioUa o Perception^, with 

Perception^ : A'^ >^ P^^ , (3) 

lpSN+{l) lpeN+{l) 

Memorizationa : Y[ 11 ''^"^ x Sa Sa, (4) 

leL\aeA, ;pgjv+(i) 

^Therefore, each level has a microscopic side: the agent behaviors, and a macroscopic side: 
the reaction function. This aspect can also be found in holonic multi-agent systems [3]. 
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Decision^ : Sa ^ V^'' . (5) 

ijsiv+co 

There is no memorization function specific to a level to preserve the coherence of the 
internal state of the agents. The environment a; of a level I produces influences through 
a function: 

Naturall : A' Y[ V^'' . (6) 

The reaction function computes next level state and time advance: 

Reaction : x r" A' x T'. (7) 

The time representation is inspired by DEVS (Discrete EVent System specifi- 
cation) [55]. T = Uigz,!''!''} denotes the time vector of the simulation, such as 
V/ G L,T' =< t^^di} >, where represents when the current event (or step, de- 
pending on the simulation model) time and dt^ its lifespan. The final simulation time 
is denoted t/. The algorithm [T] ensures the scheduling of these different functions with 
respect to temporal constraints of perception and memorization, influence production 
and reaction '23'. 

The implementation of IRM4MLS is based on the idea of micro kernel, taken from 
MadKil[^ [13j. Thus, in this approach, a technical agent, e.g., an observer or a message 
broker, would be scheduled with respect to the IR principle (cf. algo.[T|, the concept 
of level ensuring a clear separation between system and simulation agents. 

The API is minimal (seven high-level abstractions) and specifies only the methods 
needed to schedule a model (fig. [2|. Most methods are generic and then are imple- 
mented at an abstract level. Basically, to implement a model one only has to override 
perception, memorization, influence production, reaction and initialization functions. 

Agent and behaviors (such as environment and natural dynamics) are represented 
by different entities to clearly distinguish the core side of an agent, its state and mem- 
orization function, and the level sides (perception and influence production functions) 
that can change according to simulations. 

' http : //www . madkit . org 
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Figure 1: Main concepts of IRM4MLS (as a snTiplified class diagram) 



3 



Algorithm 1: simulation model of IRM4MLS 

Input: <L,Ei,Ep>,A,6,T,tf 

Output: 5{tf) 
1 while <tf do 



2 


foreach a £ A{t) do 


3 




foreach I € L : a € Ai{t) h\llp E N+{l),t'- > t'-^ do 


4 






Pa{t^) = Perceptioni{< 5^^{t^p) : Ip e N+{1) >); 


5 




end 


6 




s 


a(t' + dt'') ~ Memorizationa{pa)\ 
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end 
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foreach I e L:\fli e N+{1), < t^' V + dt^ < t^' + dt^' 
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foreach Ij e Nt (l) : t' < t^' A + dt^ > t^' do 


10 
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foreach a E Ai{t) do 
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-fi''it'') = Decision[{sait' + dt')); 
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end 


14 




end 


15 


end 
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foreach I E L : t'- + dt^ E minlt + dt) do 


17 
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end 




20 end 







3 IRM4MLS in practice: multi-level games of 
life 

3.1 Introducing a macroscopic parameter (top-down con- 
trol) 

In this section, a toy IRM4S (or 1-level IRM4MLS) model, is presented: a modified 
agent-based version of the Conway's game of life (or simply Life). This simple example 
illustrates where a macroscopic parameter, should (but should not) be introduced in 
an agent-based model that relies on the influence reaction principle: in the reaction 
function (but not in the behavioral functions of agents). Therefore, this parameter 
has a non-ambiguous semantics that does not depend on the updating scheme of the 
simulations, even in the case of strong interactior^ 

Each agent represents a cell that can be dead or alive and that has eight neighbors 

^Strong interaction implies that agents agree on the outcome of the interaction 1201 . Thus, 
such model should not be simulated with a STRIPS-like meta-model, i.e., that views an action 
as a change of the state of the system. It would lead to problems of result replication T , but 
also of parameter and result interpretation [8| I20| . 
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< < interface > > 
Simulation 



+void runSimulation(String logFilc) 
+void initO 

4-List< Agent> getAgentsO 
+List<Level> getLevels() 
+Time getInitialTime() 
+Time getFinalTimeO 



<<intcrfacc>> 
Agent 



+ Map<Lcvcl, Bcha 
+void mcmorizcO 



r> gctLcvclsO 



< < interface > > 
Behavior 



+ void pcrccivcO 

+ Map<Lovcl,Map> dociaion(List < Lcvel> li) 





<< 


into 
Nat 


facc>> 
ural 












+ Map<Le 


^cl,Map> naturalO 








1..* 








<< 

En 


into 


facc>> 
nment 




+ Map<Lcvcl 


Nat 


ural> gctLovclaO 



<<intcrfacc>> 



4-List< Agent > gctAgents() 

+ Environmont get Environment ( ) 

+ List<Level> getPerceptibleLcvcls () 

+ List<Level> getlnfluenceableLovela ( ) 

+ DynamicState getDynaniicStatc() 

4-void reactionO 



<< interface >> 
DynamicState 



+ Map gctTom 
+void addTcn 
+ Timc getTin 
+ Timc getNoj 



>rarylnfluences( ) 
3raryInfluences(Map i 



Figure 2: Java API of IRM4MLS 



in a toroidal grid. Tiie set of environmental properties is then: 

Wt,a{t) = \^ {neighbors{a),alive{a)}. (8) 

Cells evolve in parallel: the reaction function can then simply be defined as "ap- 
plying agent infiuences" (algo. [2| . If a cell is dead and has three living neighbors or is 
alive and has two or three living neighbors, it will be alive at the next step; in other 
cases, it will be dead. Let specify the behavior of the agents: 

1. they perceive the number of living cells in their neighborhood (algo. [3|, 

2. memorize their internal state, i.e., their next state (algo.|4|, 

3. and then, decide whether or not they will be alive at the next step according to 
their internal state (algo.[5|. 

The environment is static: there is no natural dynamics and thus, Naturalui returns 
0. 

One cruel and ironic aspect of Life is that a cell has generally little chance to 



remain alive in the long run (fig. 3(a) I. Moreover, what you get most of the time. 
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Algorithm 2: Reaction Algorithm 4: Memorizationg 



Input: a{t),i{t) 


Input: Pa(t),Sa(t) 




Olltnilt- s (f + rlf\ 


1 foreach a A{t) do 


1 if alive{a) A Pa(^) e {2, 3}V 


2 alive{a) = 7^(t) ; 


2 -^alive{a) A Pa{t) G {3} then 


3 end 


3 Sa{t + dt) — 1 ; 




4 else 




5 Sa(t + dt) = ; 




6 end 


Algorithm 3: PerceptioUa 




Input: 5{t) 


Algorithm 5: DecisioUa 


Output: Pa{t) 


Input: Sa{t + dt) 


1 Pa{t) = ^ alive{n) ; 


Output: 7^(t) 




1 l'a{t)=Sa{t + dt) ; 



is a board composed of small still lifes and 1-period oscillators. This behavior is 
predictable knowing the A parameter of the game of UffQ A is a complexity measure 
of cellular automata introduced by [15] that depends on the number of cell states K, 
the neighborhood and the number n of transitions to a quiescent state Sq in the 
transition function such as 

with is: = 2, TV = 9 and n = (*) + 2 ■ E,=o,i,4-8 (!) Life. 

To improve Xufe to a value A^, one needs to change the rules. In this example, 
A^ is regarded as a macroscopic parameter, explicitly introduced in the model and 
independent from cell behavior^ influences of dying cells are not taken into account 
by the reaction function with a probability p such as 



E (10) 



1=0,1,4-8 



Thus, their is a simple linear relation between p and A+: p — (A^ — A;i/e)/0.3359. 
For A^ G [0.48,0.6], the number of dying cells tends to decrease in time and large 
structures of vertical or horizontal rows eventually emerge, shaped by moving groups 
of switching state cells that seem to work at their boundaries, and eventually vanish 
when the board becomes a dense still life of density « 0.5 (fig. |4]|. 



Aii/e = 0.2734375. For A 0.25, "structures of period 1 appear. Thus, there are now three 
different possible outcomes for the ultimate dynamics of the system, depending on the initial 
state. The dynamics may reach a homogeneous fixed point consisting entirely of state Sq, or 
it may reach a heterogeneous fixed point consisting mostly of cells in state Sq with a sprinkling 
of cells stuck in one of the other states, or it may settle down to periodic behavior" 1151 p. 
17]. 

^An other macroscopic parameter, the asynchrony, has been previously introduced in Life 
is such way [5]. 
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Figure 3: Dynamics of the density of living cells, p, starting from a random grid 
with 100 replications (a) Macroscopic dynamics: expected density and variabil- 
ity of living cells in the whole grid, (b) Mesoscopic dynamics: expected density 
and variability of living cells in 10 x 10 cell clusters. 



3.2 Top-down feedback control 

In the previous example, while the macroscopic parameter has an influence on 
agents, it is not related to the state of the system and therefore, there is no need to 
observe it. The reaction function can then be viewed as an open-loop controller. In 
the next example, a top-down feedback control is introduced. 

The goal of the multi-scale model presented in this section is to keep Life boards at 
the desired density (p^), by controlling the proportion of dying cells at the mesoscopic 
level, to account for the natural variability of density between regions of the grid 
(fig. |3(b)| . Moreover, the control should affect as less as possible simulations at the 
microscopic level and, to keep it simple, should be tuned by a single linear parameter 
such as A"*" in the previous model. 

Two levels are considered: the cell (or microscopic) level, Im and the cell region (or 
mesoscopic) level, Im. At the mesoscopic level, the model properties are the expected 
density and the cells in each region: 

yt,a'^'{t) = {p+} y {cells{a'")}. (11) 

The cells behave according to the game of life rules (algo. [sj [4]and|5|. However, 
Reaction'"^ depends on mesoscopic influences (algorithmic. Ep and Ej are equal to 
{lAilm}- Mesoscopic agents have a proportional control behavior. They 

1. perceive the density of living cells in a region, 

2. memorize their internal state, i.e., the difference e between expected and actual 
densities, 

3. and then decide the influence sent to agents of Im'. 

Va'^-f G A'"Va'" G cells{a^^"),command{a''^) = kp ■ e. (12) 
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A+ 

(a) Convergence conditions (b) Simulation example 

Figure 4: (a) Mean number of steps needed to converge to a steady state starting 
from a random grid (100 replications), simulations are stopped after 2 • lO'' 
steps (simulations converge in the dark area), (b) Example of still life found for 
A+ = 0.5. 



Algorithm 6: Reaction'' 



Input: cr'"'(t),7'™'(t) 
Output: (5''"(t + 1) 
1 foreach a''" e a'™ do 



2 
3 
4 
5 
6 
7 

8 end 



rand € [0, 1[ from pseudorandom uniform distribution ; 
if command{a'"^) > rand A alive{a'"^) A ^7'a'm(i'") then 
I alive{a'^) = T ; 
else 

I alive{a'"^) = j' a'^it^"") \ 
end 



The kp parameter has to be carefuUy tuned to run realistic simulations: too small 
simulations do not achieve the desired solution (p = p"*"), too big the board density 
tends to oscillate around and the number of micro influences not taken into account 
by Reaction'"^ becomes too important. However, for appropriate kp values, this simple 
linear controller achieves good results and allows to find a good compromise between 
conflicting micro and meso knowledge (fig. (sl. 



4 Conclusion 

4.1 Discussion 

An important issue of multi-level agent-based modeling, only briefly discussed here, 
is to define the adequate methodology. Indeed, the traditional MABS methodology is 
purely bottom-up: microscopic knowledge is used to construct models while macro- 
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Figure 5: Simulation examples with kp = lO-p"*" and initial density p(to) — . 
r represents the rate of microscopic influences not taken into account (in %) by 

Reaction!'^ . 



scopic knowledge is used to validate models [^. Thus, it seems irrelevant in a multi- 
level context. Three general conclusions can be drawn from the previous examples: 

• a parameter should be introduced at its observation level in the model; therefore, 
each observed level should be explicitly represented in the model, 

• an intuitive way to model an external control on a level I is to modify the reaction 
function of I, i.e., to modify the way influences of agents of I are taken into 
account (but not agent behavioral functions), according to external influences; 
an external feedback control implies both observation and influence relations: 



• such a controller can be viewed as a technical tool that aims to find a compro- 
mise between conflicting knowledge from the different studied levels and achieve 
realistic simulations from different studied points of view [16]. 

Pattern oriented modeling (POM) consists in "the mulU- criteria design, selection 
and calibration of models of complex systems" [12]. Many aspects of this methodology, 
developed in the context of ecology, seem particularly relevant for multi-level agent- 
based models since "patterns" are generally observed at different levels of organization 
in complex systems. However, the problem is far from being solved. For instance, 
the introduction of a dynamic level of detail raises several questions regarding, e.g., 
the validation of the model or the representation of composite agents [251 127) . A 
case study of three real world multi-level agent-based models reveals other interesting 
methodological issues |11] . 

4.2 Related works 

At least three works could be related to this one: 

• ML-DEVS is an extension of DEVS that allows the simulation of multi-level 
models (and not only coupled models in which the behavior of a model is deter- 
mined by the behaviors of its sub-models) [H]. Two types of relation between 



N-{l)^N-{l), 
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levels are defined: information propagation and event activation which are quite 
similar to those defined in IRM4MLS. However, ML-DEVS supports only pure 
hierarchies of models, i.e., interaction graphs are viewed as trees [TS|. DEVS, as 
a generic event-based simulation framework, has also been extended to support 
agent-based models [24]. A major design difference between IR and DEVS based 
approaches is the technical orientation of the latter leading to an important gap 
between conceptual and computational models. 

• PADAWAN (Pattern for Accurate Design of Agent Worlds in Agent Nests) 
is a multi-scale agent-based meta-model based on a compact matricial repre- 
sentation of interactions: lODA (Interaction-Oriented Design of Agent simula- 
tions) [141 129) . Moreover, authors analyze the structure of what is called here 
interaction graphs in multi-scale models (a relevant issue for IRM4MLS as well), 
and conclude they should be viewed as upper semilattices and not simply trees 
as suggested elsewhere. A major design difference between IR and lODA based 
approaches is that the latter constraints the definition of interactions, leading 
to a simple but restrictive simulation framework. 

• GAMA^is a MABS platform with a dedicated modeling language, GAML, that 
offers multi-level capabilities [32] ■ Moreover, it includes a framework (a set 
of predefined GAML commands) to agentify emerging structures ^3]. It is 
certainly the most advanced platform, from an end-user point of view, that 
integrates a multi-level approach. 

4.3 Perspectives 

The main perspectives of this work concern the implementation of existing works with 
IRM4MLS: 

• the concept of Poly Agent [261 128j . 

• multi-level organizational models widely used in engineering sciences such as 
holonic multi-agent systems (cf. footnote 2), system of systems and heterarchical 
control |4l [T7l[22] . 

• multi-scale tools: generic scaling operators and emergence detection and reifica- 
tion algorithms [3l 125] . 

Moreover, the first model presented in this paper could be used to explore the 
relations between computational capabilities of a cellular automaton (A^), noise (a 
function of A"*" — A) and entropy. Moreover, finding the conditions under which cells 
arrange themselves in a steady state could be an interesting way to solve heuristically 
large instances of the maximum density still life problem ,7]. 

The second model illustrates a form of simple proportional top-down feedback 
control. Such approach could be generalized to model more complex of cross-level 
feedback control, using integrations and derivates of observed variables. 
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